Research on Text Comprehension in Multimedia Environments

Based on underlying theories of L2 reading comprehension and text comprehension with multimedia, and in light of the new tools and modes of information presentation that are now available, researchers have a variety of avenues for studying how people comprehend text in a second language with the help of multimodal instructional materials. In this paper we discuss how L2 reading research is focusing increasingly on the cognitive processes involved in reading, that is, the interaction of lower-level, bottom-up processes such as vocabulary acquisition with higherlevel, top-down processes such as activating prior knowledge. We combine this knowledge with current research on learning with multimedia, focusing on how learners integrate verbal and visual information, particularly with respect to the individual differences among learners that moderate, if not determine, learning processes. INTRODUCTION The process of reading and comprehending in one's native language (L1) is very complex, due to the myriad of factors that interact with each other in a non-linear and non-sequential manner. The issues and their accompanying complexity are further compounded when describing and understanding reading comprehension in a second or foreign language (L2). In facilitating L2 reading comprehension, the use of sound, pictures, and animated pictures or video in addition to text have played an important role in vocabulary acquisition and in overall text comprehension, and are unquestioned components of instructional materials for language learning (Chun & Plass, 1996a, 1996b; Cohen, 1987; Hanley, Herron, & Cole, 1995; Leow, 1995; Oller, 1996; Omaggio, 1979; Secules, Herron, & Tomasello, 1992). The possibility of an instructional use for these different modes of information on a computer raises questions concerning learning from media (Clark, 1994; Clark & Salomon, 1986; Kozma, 1991; Ross, 1994), and concerning the specifics of language learning with multimedia. Of special interest are questions regarding the difference in cognitive processes in learning from different sources, and regarding the effect of individual learner differences on learning from media. How does the process of comprehension of text differ from comprehension of pictures, and how does the combination of pictures and text affect comprehension? What effect do students' learning preferences, abilities, and cognitive styles have on the answers to these questions? This paper is concerned with the research regarding text comprehension in multimedia environments. We first summarize research results in L2 reading comprehension and text comprehension with multimedia, and synthesize these findings into a model of L2 reading comprehension with multimedia. We then review the research questions currently asked by scholars working in the field and, based on the model proposed, derive new and more specific research questions. Volker Hegelheimer & Carol Chapelle Methodological Issues in Research on Learner Computer ... Language Learning & Technology 56 L2 READING COMPREHENSION Historical Perspective From a historical perspective, our understanding of reading in a second or foreign language (L2) has changed considerably in the last several decades. In the midto late 1960s, reading was considered a skill for learners to acquire, mainly to reinforce the grammar and vocabulary being taught orally (e.g., under audiolingualism). In the 1970s researchers argued that greater importance should be placed on reading and advocated a psycholinguistic model or theory of reading (Goodman, 1967; Smith, 1971, 1979). During the 1980s the perspectives of Goodman and Smith were extended, and L2 researchers began to examine L1 reading research more closely (e.g., Bernhardt, 1991; Goodman, 1985; Smith, 1982). Currently, there are two overlapping approaches to viewing and describing reading that reflect, in part, the development of reading research. The first might be termed a "reading components" perspective in that reading is subdivided into skills and knowledge areas. This approach goes well beyond viewing reading as a simple collection of skills or knowledge in that it also focuses on the cognitive processes involved in reading in general. The second approach to describing reading is in terms of metaphors: The most prevalent metaphors in the literature are the bottom-up approaches, the top-down approaches, and the interactive approaches. Due to the fact that many of the current views of L2 reading have been shaped by research on L1 readers, this section first summarizes the main characterizations of L1 reading comprehension. The additional factors that come into play when dealing with L2 reading comprehension will be discussed next. Finally, a comparison of L1 and L2 readers in terms of cognitive comprehension processes will lead into the following section on text comprehension with multimedia. Reading Components Approach Briefly, the view of reading from a so-called "reading components" perspective proposes to subdivide reading into six general component skills and knowledge areas (as summarized by Grabe, 1991): (a) automatic recognition skills; (b) vocabulary and structural knowledge; (c) formal discourse structure knowledge; (d) content/world background knowledge; (e) synthesis and evaluation skills/strategies; and (f) metacognitive knowledge and skills monitoring. Some researchers rank these factors in terms of importance for the comprehension process. For example, Laufer & Sim (1985) posit the following factors in order of decreasing importance: knowledge of vocabulary, subject matter, discourse markers, syntactic structure. In essence, they find that vocabulary is most important, syntax least important. It is important to note that the above-mentioned skills are increasingly being described by cognitive and educational psychologists in terms of the processes that learners go through in developing such skills or knowledge. For example, the so-called "automatic recognition skills" that typically involve recognizing letters, characters, and words have recently begun to be recognized as important in L2 reading. Automatic lexical access involves lower-level processing and, in a fluent reader, requires little processing capacity (cf. Koda, 1992). With the shift in emphasis from a pure skills approach to a process approach, researchers turned to the comprehension research that formed a major domain of the field of cognitive psychology. Thus, most of the current views of L2 reading have been shaped by the solid and extensive body of research by cognitive psychologists on L1 reading comprehension (Grabe, 1991). There are a number of excellent reviews of the literature on L1 reading comprehension (e.g., Bernhardt, 1991; Grabe, 1991; Horiba, 1996; Swaffar, Arens, & Byrnes, 1991). As an overall description of the process of reading comprehension, Swaffar et al. state that "fluent readers synthesize textual subsystems (e.g., content, context, intent, language) into a larger metasystem of meaning" (p. 21) and that "in short, readers comprehend a text Volker Hegelheimer & Carol Chapelle Methodological Issues in Research on Learner Computer ... Language Learning & Technology 57 when they construct a mental representation for incoming pieces of verbal information" (p. 22), see section "Text Comprehension with Multimedia." De Beaugrande (1982), among others, concludes that "what is in fact comprehended is not sentences, but conceptual content" (p. 180). One cognitive view of reading suggests that it is a process or set of ordered stages, consisting of a beginning state, an end state, and intervening transformations (e.g., Just & Carpenter, 1987). Some researchers believe that reading comprehension involves multiple cognitive processes that are hierarchically related to one another (e.g., Horiba, 1996). These processes (from recognizing letters, characters, and words, to analyzing the syntactic and semantic structure of clauses and sentences, to generating inferences) must take place in an orchestrated manner (Rayner & Pollatsek, 1989). However, neither the "hierarchy of processes" view nor the proposition that they take place in an "orchestrated manner" insists that the process be linear. Just and Carpenter (1980) define the process as a clearly nonlinear one. Readers are active in the selection of portions of the text for processing, and former portions of the text may inform latter ones, just as latter portions of the text may inform former ones through feedback. Similarly, other researchers believe that reading comprehension results from interactive variables that operate simultaneously rather than sequentially (Samuels & Kamil, 1984; Swaffar et al., 1991). A second major approach to understanding the reading process is to describe it using metaphors. The three most common metaphors for describing the reading process are bottom-up processing, top-down processing, and interactive processing (cf. Samuels and Kamil, 1984; Silberstein, 1987; Swaffar et al., 1991). These terms and metaphors are described below. Metaphoric Approaches Bottom-up processing models place primary emphasis on textual decoding. They can be seen as datadriven and emphasize the priority of the text as input and, hence, lower-level processes such as letter and word recognition. In contrast, top-down models place primary emphasis on reader interpretation and prior knowledge. They are seen as concept-driven, in the sense that the text is "sampled" and predictions are made on the basis of the reader's prior syntactic and semantic knowledge (Goodman, 1967). Other models are more balanced in suggesting that linguistic knowledge from several sources (orthographic, lexical, syntactic and semantic) interacts in the L1 reading process (Rumelhart, 1977). Readers may also try to compensate for deficiencies at one level (e.g., word recognition) by relying more on a source at a lower or higher level (e.g., contextual knowledge) (Stanovich, 1980). Texts are described in terms of their linguistic properties, their content and organization, and whether or not they are authentic texts or discourse. Readers are described in terms of their linguistic competence, their background or world knowledge, and affective factors (e.g., learning styles, motivation) (cf. Swaffar et al., 1991). A third metaphor posits interactive processing in seeking to explain the reading process. The term interactive approaches can refer to two different conceptions: (a) to the general interaction between reader and text, that is, the reader makes use of information from his/her background or prior knowledge in (re)constructing the text information; and (b) to the interaction of many component skills that work together simultaneously in the process. "Simply stated, reading involves an array of lower-level rapid, automatic identification skills and an array of higher-level comprehension/interpretation skills" (see Grabe, 1991, p. 383; Williams & Moran, 1989). Interestingly, most cognitive psychologists stress the interaction-of-skills arrays whereas most L2 researchers stress the interaction between reader and text. In sum, an interactive approach to reading is one that takes into account the contributions of both lower-level processing skills (identification or decoding) and higher-level comprehension and reasoning skills (interpretation and inferencing). It is widely believed that comprehension results from these interactive variables operating simultaneously rather than sequentially. Volker Hegelheimer & Carol Chapelle Methodological Issues in Research on Learner Computer ... Language Learning & Technology 58 Volker Hegelheimer & Carol Chapelle Methodological Issues in Research on Learner Computer ... Language Learning & Technology 59 Reading Problem or Language Problem? We now turn to some of the main factors that influence L2 reading but that are not normally considered in L1 research, namely: (a) differences in background knowledge, that is, L2 readers start with a smaller L2 vocabulary knowledge than L1 speakers possess when beginning to read in their native language, but L2 readers start with greater world knowledge; (b) differences in language processing, that is, the transfer effects or interference from L1 to L2 on the orthographic, lexical, syntactic, and discourse levels (cf. Koda, 1993); and (c) differences in the social context concerning literacy, that is, expectations about reading and how texts can be used (Grabe, 1991). 1 In considering the factors listed above, one of the most prevalent questions in recent research is: Is L2 reading more a reading or a language problem? (cf. Taillefer, 1996). The question was first posed by Alderson (1984) who states it is both, "but with firmer evidence that it is a language problem, for low levels of foreign language competence" (p. 24). After a decade of research, the question of whether L2 knowledge or L1 reading would be the better predictor of second language reading performance was revisited by Bernhardt and Kamil (1995). They note that the two hypotheses most commonly associated with L2 reading are in conflict because of the either/or nature of the question. The first hypothesis, initially termed the "short circuit hypothesis," now referred to as the Linguistic Threshold Hypothesis (LTH), states: In order to read in a second language, a level of second language linguistic ability must first be achieved. In other words, language is the key factor in reading. The second hypothesis is referred to in the recent literature as the Linguistic Interdependence Hypothesis (LIH), stating that: Reading performance in a second language is largely shared with reading ability in the first language. The idea is that once the ability to read has been acquired, it can be transferred to a second language. Bernhardt and Kamil (1995) reviewed studies since 1984, analyzed their own new data, and found considerable consistencies "in the amount of variance accounted for by first language literacy (upwards of 20 per cent). However, linguistic knowledge is consistently a more powerful predictor (upwards of 30 per cent)" (p. 15). Based on the results of their study, they conclude that the question is not unambiguously answerable. They instead propose to restate the problematic and ask both: (1) How first language (L1) literate does a second language reader have to be in order to make the second language (L2) work? and (2) How much second language (L2) knowledge does a second language reader have to have to make the first language (L1) reading knowledge work? It then becomes important to consolidate both LIH and LTH in order to account more precisely for the remaining 50 percent of the variance in L2 reading that is yet to be explained. The apparent interdependence of L1 and L2 abilities leads to the question of how L1 and L2 readers compare in terms of cognitive comprehension processes (cf. Davis & Bistodeau, 1993). Are there two separate, yet parallel cognitive processes at work, or are there generic language, and more specifically, generic discourse processing strategies that accommodate both first and second languages? (Bernhardt, 1991). Beginning L2 readers are thought to focus on low-level process strategies (e.g., word identification), whereas more proficient readers shift attention to more abstract conceptual abilities and make better use of background knowledge; that is, they use textual information to confirm and predict the information in the text (Coady, 1979). Recent evidence has in fact shown that L1 readers use much of their attention for higher level processes such as the generation of inferences and general knowledge associations, more so than L2 readers, who pay more attention to lower level processes (Horiba, 1996). In conclusion, although the broad goal of this paper is to better describe and understand the components and processes in text comprehension, it is not feasible to discuss in detail all of the factors and perspectives listed above. We will thus propose a model of the basic components and processes in L2 reading as visualized in Figure 1, and acknowledge the numerous other moderating factors under the Volker Hegelheimer & Carol Chapelle Methodological Issues in Research on Learner Computer ... Language Learning & Technology 60 rubric of individual differences (e.g., prior knowledge; learning preferences; spatial, verbal and linguistic ability). Our model focuses on the relationship between a microor lower-level component, vocabulary knowledge, and the macroor higher-level component of propositional comprehension. It is hypothesized, first, that proficient readers utilize both bottom-up and top-down processing, and that successful comprehension is the result of an interaction and smooth transitions between both types of processing (cf. Coady, 1993; Silberstein, 1987; Swaffar et al., 1991), and second, that presenting information in multiple modes can facilitate the different types of comprehension processes. The latter will be discussed in more detail in the following section. TEXT COMPREHENSION WITH MULTIMEDIA Multimedia environments allow for the addition of visual and auditory information to a text in order to improve comprehension. In this section, we will first summarize the cognitive processes involved in text comprehension without these types of additional information. Based on the differences between comprehending text and comprehending graphics, we will then expand our model by identifying the function of multimedia information as aids to text comprehension. As discussed briefly in the preceding section, the process of text comprehension in the first language (L1) is described in current models as an active process of constructing mental representations of the text information. This involves first the construction of a mental representation of the text's linguistic surface structure, then the construction of a propositional representation of the semantics of the text, and eventually the construction of a mental model of the subject matter described in the text with the help of existing cognitive schemata (Carrell, 1984b; Johnson-Laird, 1983; Schnotz & Grzondziel, 1996; van Dijk & Kintsch, 1983). The term "propositional representation" is used to describe subject matter in a hypothetical mental language in single statements each of which is either true or false. A "mental model" is "a mental representation consisting of parts that interact with one another according to principle-based rules" (Mayer, 1993, p. 242). Examples for mental models are a mental model of a meteorological system or, in the case of literary texts, a mental model of settings, events, and characters in a novel. When reading a text, mental models are created based on propositional representations, using prior knowledge and experiences in the form of existing cognitive schemata (i.e., descriptions of typical objects and scenarios). The resulting mental model is a hypothetical internal quasi-object that represents subject matter based on a structural or functional analogy between the subject matter and the model. Like schemata, it also represents typical objects or scenarios. The mental model can then be used to derive new information regarding the subject matter. The difference between learning from text and learning from pictures results from the different types of representations of knowledge: Text represents information in symbolic structures of a language and is processed sequentially, that is, word by word or sentence by sentence (Schnotz, 1993; Schnotz & Grzondziel, 1996). Pictures, on the other hand, convey their information by means of a visuo-spatial structure (i.e., the spatial arrangements of the components of the picture), and thus represent the subject matter by employing an analogy based on common structural properties (Clement & Gentner, 1991; Gentner, 1983; Johnson-Laird, 1983; Schnotz, 1993; Vosniadou & Ortony, 1989) and encode information in parallel or simultaneously (Clark & Paivio, 1991). Schnotz (1993) argues that the construction of a mental model of subject matter is qualitatively different in learning from text and learning from images. He argues that "graphics, like mental models, possess also inherent structural properties used for their representational functions, which is not the case with text" (p. 248). Consequently, while the construction of a mental model from text requires the construction of Volker Hegelheimer & Carol Chapelle Methodological Issues in Research on Learner Computer ... Language Learning & Technology 61 propositional representations, which then have to be integrated into the mental model, the use of images "provides the possibility of a relatively direct construction of a mental model" (p. 248-249), since they, like mental models, employ analogies to represent the subject matter. In other words, text comprehension requires an indirect transformation between the symbolic representation of the text and the analog mental model, while the comprehension of an image requires establishing an analogy between the picture and the corresponding mental model. This bypasses the propositional representation of information altogether, and is thus assumed to be language independent. The question, then, is: How can we combine both types of representations to improve comprehension of a text? Multimedia Aids for Text Comprehension The model described above views text comprehension as the process of transforming a symbolic representation of information into a propositional representation, and then, based on cognitive schemata, into an analog mental model. How can these processes be supported by additional sources of information? Mayer (1984) names three types of aids for text comprehension: (a) aids for selecting information; (b) aids for building internal connections; and (c) aids for building external connections. Aids for selecting information serve mainly to focus the reader's attention on certain aspects of the target information and thus improve the chance that this information is processed. Following the proposed model, this would assure that a propositional representation of the information is in fact constructed. Aids for building internal representations are designed to support the reader's building of internal connections among the units of information presented, that is, organizing the presented information into a coherent structure of the logical relations among idea units in the text. This will help assure a coherent structure of the propositional representations, based on cognitive schemata. The third type of aids for text comprehension, aids for building external connections, are designed to help the reader build connections between the ideas in the text and an existing mental model, thus integrating these new ideas into the existing mental model (Mayer, 1984; Resnick, 1982). These aids support the construction and extension of the mental model based on the propositional representations. The introduction of aids in presentation modes other than text can be based on these three types of aids for text comprehension. Although the process of learning from different sources of information, using different types of representations of the subject matter, can engage the learner in qualitatively different cognitive processes, it is ultimately the function of the three different types of aids in the process of text comprehension that is of interest. For example, the integration of new information into the existing mental model can be supported with a concrete advance organizer (Ausubel, 1960; Mayer, 1984). This advance organizer could be presented either as text or as a video. Although the cognitive processes involved in comprehending the text are qualitatively different from those involved in comprehending the video, the function of the concrete advance organizer as support for the integration of information into the mental model remains unchanged, irrespective of the presentation mode. With the use of information in multiple presentation modes, the aids for text comprehension could conceivably be presented in textual form, in visual form, in auditory form, or in any combination of these presentation modes. It should be emphasized that the processing of the material is the focus of our investigation, not the stimulus material provided to the learner. While spoken text as an auditory form of presenting information is perceived by the learner through a different perceptual channel than a written text, it is like written text in using a symbolic representation of information, and thus involves similar cognitive processes of text comprehension. Visual information, on the other hand, is an analog representation of information that can be directly mapped onto the mental model by establishing an analogy between the visual information and the corresponding mental model (Gentner, 1983; Schnotz & Grzondziel, 1996). Volker Hegelheimer & Carol Chapelle Methodological Issues in Research on Learner Computer ... Language Learning & Technology 62 Applying Mayer's (1984) aids for text comprehension to multimedia aids, Figure 2 extends our proposed model by proposing the different functions multimedia information can have to aid text comprehension: It can support the processes of selecting information, of building internal representations (organizing the presented information into a coherent structure), and of building external connections (integrating these new ideas into the existing mental model). In summary, learning from text and learning from pictures are qualitatively different ways of constructing mental models. While text comprehension is an indirect transformation between a text as a symbolic representation and the mental model as an analog representation, picture comprehension is a direct mapping of the picture as an analog representation onto the mental model as an analog representation by establishing an analogy between the visual information and the corresponding mental model (Schnotz, 1993; Schnotz & Grzondziel, 1996). These qualitative differences can be used to aid text comprehension, namely, in supporting the processes of selecting information, organizing the presented information into a coherent structure, and integrating these new ideas into the existing mental model (see Figure 2; Mayer, 1984). In the following section we will further describe our proposed model of text comprehension in multimedia environments by specifying the function of multimedia information to support the cognitive processes involved in text comprehension. Proposed Model of Text Compprehension in Multimedia Environments A number of models address the collaborative effects of spatial and verbal processes to enhance comprehension and recall. The model proposed here integrates and expands some of these existing models (cf. Figure 1). On a micro level, the enhancing effects of the use of verbal and nonverbal (e.g., visual) presentation modes of information on vocabulary acquisition can be interpreted in terms of dual coding (Paivio, 1971, 1986). The dual coding theory postulates the existence of two different storage systems for information--a verbal and a nonverbal system. Information in a symbolic representation is stored in the verbal system, information in a nonverbal, analog representation is stored in the nonverbal system. These systems are defined to be independent, allowing for additive effects in recall if information is coded dually (i.e., in both systems). In an extension of the dual coding theory to multimedia learning, Mayer and Sims (1994) found that the contiguous presentation of visual and verbal material made it more likely for the learner to build referential connections between the visual representation and the verbal representation in short-term memory, which resulted in better performance in transfer tasks than the successive presentation of the materials, especially for high-ability students. The advantages of storing information in two different systems are: (a) a more elaborate encoding, resulting in more retrieval routes to the material; (b) the possibility of storing more information in these two systems; and (c) the opportunity to store information in the optimal system for the specific type of information, for example, symbolically coded information in the verbal system, or information coded in an analog form in the nonverbal system (Kirby, 1993; Paivio, 1971, 1986). On a macro level, text comprehension with multimedia can be interpreted with a generative theory of multimedia learning (Mayer, 1997). The generative theory of multimedia learning views the learner as knowledge constructor who actively selects relevant words and images from the information presented, organizes these words and images into coherent mental representations, and integrates the newly constructed visual and verbal representations with one another (Mayer, 1997). These cognitive tasks are based on cognitive abilities and cognitive styles of the individual that have an influence on the preference for using one type of information over another, and on performance when the learner is allowed to use Volker Hegelheimer & Carol Chapelle Methodological Issues in Research on Learner Computer ... Language Learning & Technology 63 either visual or verbal information, or a combination of both, for learning. The apparent effect of individual differences on learning from different sources will be discussed later in this paper. In summary, we postulate that different cognitive processes are involved in micro level processing and macro level processing of multimedia information (cf. Figure 1). On a micro level (e.g., vocabulary acquisition), the presentation of visual information contiguously with verbal information results in the construction of referential connections between the verbal and the visual mental representations of the material, and the storage of the information in two different systems, a verbal and a nonverbal system (dual coding). On a macro level (e.g., overall text comprehension), visual information serves as an aid for text comprehension and functions as supplemental information that is added to the mental model of the text by mapping the analog visual representation onto the analog mental model. The visual information can aid in text comprehension in three different functions: (a) in selecting information, (b) in organizing the selected information into a coherent structure of propositions using cognitive schemata, and (c) in integrating these propositions into the mental model. Consequently, visual material to support vocabulary acquisition has to be designed differently from visual material to aid text comprehension, depending on the cognitive processes to be supported. It can be expected, however, that under some circumstances the use of different presentation modes of information can have deleterious effects in the processing of the information. This will be the issue of the following section. Deleterious Effects of Learning with Multimedia As described in the last section, studies have shown that collaborative effects of processing of verbal and visual information can enhance learning. Other studies, however, found that there can also be competitive effects of learning from different sources, such as task interference between modes of processing, the directing of attention, and learners' individual differences. Task interference can occur in complex tasks when the visual and verbal processes are not automated but require executive resources, and when they have to be executed under time pressure (Kirby, 1993). For example, when viewing a film in a foreign language, a learner may not have enough executive resources to comprehend the foreign language and all the visual information given. Similarly, information in different presentation modes that are perceived through the same channel can compete for perceptual resources (Mayer & Anderson, 1991). For example, an animation (visual presentation mode) may be accompanied by a text (verbal presentation mode). If both pieces of information have to be perceived visually, though, (e.g., when the text is printed), then the learner may have to split attention between both types of information, which may have deleterious effects on learning. However, Mayer and Anderson (1991) found that when the text information is presented as voice over, using a perceptual modality other than the animation, learning and transfer can be improved. When presented with both visual and verbal information, the learner may direct his or her attention to the type of information perceived as more important or more interesting, away from the other mode which may in fact contain more important information. This assumption on the part of the learner can for instance be based on interest, on the perceived demand characteristics of a source, or on the learner's perceived self efficacy in processing material from a particular source, and results in a different amount of invested mental effort (Kirby, 1993; Salomon, 1983; see Figure 1). Animated pictures, for instance, convey in addition to the visuo-spatial structure of static pictures information about the transformation of the structure over time. This can be seen as an aid to create dynamic, runable mental models of the information presented. The use of animated pictures or video, however, may lead to a lower amount of invested mental effort than the use of static pictures (Chun & Plass, 1996a; Salomon, 1983; Schnotz & Grzondziel, 1996). Salomon (1983) found that children invest Volker Hegelheimer & Carol Chapelle Methodological Issues in Research on Learner Computer ... Language Learning & Technology 64 less mental effort in comprehending materials on television than they invest in materials in print, and perceive materials presented on television as shallower and less variable than materials presented in print. In a more recent study, Schnotz and Grzondziel (1996) compared learning from static pictures and animated pictures of time zones of the earth and found that learners who had access to the animated pictures did not perform better in questions requiring the use of mental simulations. They conclude that the use of animated pictures can result in a more superficial processing of the subject matter than the use of static pictures. Chun and Plass (1996a) found a similar effect for L2 vocabulary acquisition, where students who used the text and picture annotations scored higher on the follow-up vocabulary test than students who used text and video annotations. In the preceding sections of the paper we have repeatedly mentioned the moderating effects of individual differences on learning and performance. In the following section we will take a closer look at some of these learner variables that are especially related to text comprehension in multimedia environments. INDIVIDUAL DIFFERENCES A large number of studies indicate that individual differences can have a moderating effect on L2 reading comprehension, as well as on learning from information presented in different modes. Salomon (1989), for instance, points out that text and pictures affect learning for the cognitive functions they can and do perform, and that individual differences, among other factors, do not only moderate, but often determine how stimuli are perceived and processed. These differences include, but are not limited to, linguistic competence, prior knowledge, learning styles, cognitive styles (e.g., reflection/impulsivity and field dependence/independence; cf. Jamieson, 1992), abilities (general, spatial, verbal), strategies (cf. Anderson, 1991; Kern, 1989; Raymond, 1993), and affective factors, such as interests, motivation, and attitude (cf. Gardner, Day, & MacIntyre, 1992). While most of these learner variables have an effect on learning and performance in general, some are especially associated with L2 reading comprehension and with learning in multimedia environments. Examples are verbal and spatial abilities, visualizer and verbalizer learning preferences, and background knowledge. Since space prevents a comprehensive review of the subject, we will focus on these individual differences and their effect on second language acquisition and especially L2 text comprehension with multimedia. A number of books and articles offer a more thorough review of individual differences in general than will be given here (e.g., Ehrman & Oxford, 1989, 1990; Jonassen & Grabowski, 1993; Keefe, 1989; Reid, 1987; Skehan, 1991). Verbal ability. While verbal ability has been shown to have effect on text comprehension (Knight, 1994), some studies also found it to be a predictor for the effectiveness of visual aids. Levie and Lentz (1982), for instance, concluded in a review of research on illustration effects that visual aids may be more helpful to learners with low verbal ability than for those with high verbal ability. Peek (1993) suggests that poor readers may be less capable of building mental representations based on the text itself, but may with the help of visual aids be able to build the appropriate representations. Spatial ability. Denis (1982) found in a series of four experiments on text comprehension that high imagers, who tend to elaborate images that express the content of the text they are reading, spend more time reading and have a better recall of a descriptive text that evoked imagery than low imagers who tend not to imagine while they are reading. Using an abstract, nonimageable text, no differences between the groups in reading time or recall were found. A number of studies of attribute x treatment interactions (ATI) show, for example, that multimedia effects are strongest for low-prior knowledge and high-spatial ability students (Mayer & Gallini, 1990; Mayer, Steinhoff, Bower, & Mars, 1995; Mayer & Sims, 1994). Volker Hegelheimer & Carol Chapelle Methodological Issues in Research on Learner Computer ... Language Learning & Technology 65 Visualizer/Verbalizer. Studies of ATI effects involving learning preferences (e.g., the visualizer/verbalizer preference), found a moderating effect of these preferences both on vocabulary acquisition (Chun & Plass, 1996a) and L2 reading comprehension (Chun & Plass, 1996b). While students who reported remembering verbal annotations did not remember more definitions overall than did students who reported remembering visual annotations, the analysis revealed a significant interaction of learning preferences and annotation type, in which visualizers were more likely to correctly produce a definition when they reported using a visual retrieval cue (i.e., being reminded of a corresponding picture or video) than when they reported using a verbal retrieval cue (i.e., being reminded of reading a text definition). Similarly, verbalizers were more likely to correctly produce a definition when they reported using a verbal retrieval cue rather than a visual one. These results suggest that visualizers are more effective in using visual cues for remembering vocabulary information whereas verbalizers are more effective in using verbal cues. For text comprehension, Plass, Chun, Mayer, and Leutner (1997) found that visualizers performed better on propositions for which both visual and verbal annotations were provided than on those for which only verbal annotations were provided, whereas verbalizers performed well on both types of propositions. Similarly, the effect of a visual preview of the text was strong for visualizers but weak for verbalizers. Content and background knowledge. Proponents of top-down approaches to reading refer to a large body of research on how prior knowledge affects reading comprehension and, in particular, to evidence that both content schemata and cultural background information facilitate comprehension. Schema theory for L2 reading has been investigated extensively and appears to be a very useful notion for describing how prior knowledge is integrated in memory and used in higher-level comprehension processes (Carrell, 1984b, 1987; Carrell & Eisterhold, 1983; Anderson & Pearson, 1984). The theory provides a strong rationale for both prereading activities and comprehension strategy training, that is, that readers need to activate prior knowledge of a topic before they begin to read and that this activated knowledge facilitates the reading process (Carrell, 1985, 1988). With regard to the model proposed, the activation of prior knowledge using multimedia aids, such as visual advance organizers, is one method of supporting the process of integrating newly acquired knowledge with an existing mental model (cf. Hanley et al., 1995). In summary, an increasing number of studies can be found that investigate ATI effects involving individual differences as attributes and different multimedia treatments. According to Salomon's (1972) preference model, one of the potentials of multimedia is to accommodate these individual differences and provide learners with the materials they need (Chun & Plass, 1996a, 1996b; Leutner, 1992; Plass et al., 1997). In the following section we will draw a framework of research questions that are based on the belief that individual differences are one of the most important factors in studying L2 text comprehension in multimedia environments.